Building Statistical Appearance Models Using Residual Information

Statistical appearance models are generated by applying statistical analysis on the color and shape variation in an ensemble of image objects belonging to the same class. Statistical appearance models have proved useful for locating, reconstructing and interpreting image objects that undergo systematic appearance variation. In this paper we demonstrate how residual information can be incorporated in the model building process in an attempt to generate models robust to local occlusions and models that can deal effectively with subtle types of variation. With our approach we first train a statistical appearance model using the standard technique and then estimate the residual obtained by reconstructing the samples from the training set. The analysis of the residual information allows us to define the areas of the object that the model fails to model correctly. Further statistical analysis is applied locally to such areas of the object. Quantitative results prove that statistical appearance models built using the proposed method are capable of reconstructing more accurately the appearance of previously unseen objects belonging to the same class.